Range imaging systems, also referred to as range cameras, range imaging cameras or ranging devices, determine both distance and intensity for objects in a scene to be imaged. One common type of range imaging camera utilizes Amplitude Modulated Continuous Wave (“AMCW”) technology.
As discussed in more detail below, a range imaging camera utilizing AMCW technology typically uses a light source to emit an intensity modulated illumination signal to illuminate a scene to be imaged. The signal is then reflected back to the range imaging camera by objects within the scene. Multiple images of the reflected light are then captured by a specialized sensor chip, such as a CMOS image sensor. A processor is then used to compare these multiple images to each other and to the emitted intensity modulated illumination signal to determine phase changes in the intensity modulation of the light. These phase changes are then analyzed to determine the distance between the objects and the range imaging camera, as well as to determine the size, shape and location of the objects in the scene. Range imaging cameras are therefore very useful in many varied applications such as automotive safety and control systems, multimedia user interfaces, and image segmentation and surface profiling.
Effective determination of the range between a range imaging camera and an object to be imaged depends on the phase difference between the intensity modulated illumination signal and the received signal. Because the image sensor is generally modulated with the illumination signal, as light travels away from a range imaging camera, that light begins to slip out of phase with the modulation of the image sensor. Consequently, the shorter the distance traveled by the light, the more in-phase the light will be. In other words, light reflected from objects close to the range imaging camera have an intensity modulation that is close to in phase with the modulation of the range imaging camera, while light reflected from distant objects has an intensity modulation that is out of phase with the range imaging camera. While the same signal is commonly used to modulate both the illumination signal and the image sensor, different signals may be used in some situations to achieve similar results.
As with a digital photograph, the captured images will typically be formed by an array of pixels, with objects in the scene being effectively mapped onto the pixel array. Unlike a typical digital image, the pixels captured by a range imaging camera will record not only the illumination intensity (which may include the color) of the light reflected from an object in a scene, but also a phase of the modulation envelope. Accordingly, each of the pixels in a captured image will have a phase component that is based on the light reflected back to that pixel by objects in the scene. Furthermore, pixels that correspond to areas of the scene that do not have objects in them will not receive a reflected signal, but will only receive ambient light or light reflected from another source (such as the sun) or multi-path modulated light (emitted light reflecting off other objects in the scene creating an undesired interfering signal). Typical range imaging camera sensors include background or DC rejection circuitry that allows more efficient use of the sensor's dynamic range for capturing the modulated light, therefore, background light is effectively ignored by the sensor.
While there are many ways to implement typical range imaging cameras, FIG. 1 has been provided to show a simplified version of one such range imaging camera. As shown in FIG. 1, a typical AMCW range imaging camera 100 may include a signalling device 101 to emit an intensity modulated illumination signal, e.g., a laser diode or a light emitting diode, towards a scene to be imaged and an image sensor 102 to capture images using the portions of the emitted signal that are reflected back to the system 100 from the objects within the scene. A processor 103 then compares the emitted signal with captured images to determine the intensity and the phase shift for each pixel in the captured image. By doing so, the processor is able to determine the distance objects in the scene are from the system 100. The system 100 may optionally contain an input device 104 to control the system 100, a display 105, and a memory 106; or these devices 104, 105, and 106, may be in the form of a separate computer system connected by an interface cable. These devices may be chosen from among any of the known devices for performing such functions. There are many known ways of modulating the signalling device 101 and the image sensor 102. For example, the signalling device may simply be a light source that is cycled on and off very quickly, while the image sensor 102 may be modulated using a high speed shutter or by controlling the gain or sensitivity of image sensor. The processor 103 may also include a signal generator, or a similar device, to assist in modulating the signalling device 101 and the image sensor 102. However, a signal generator may also be separately provided.
FIG. 2 depicts a simplified functional diagram of the standard range imaging camera described above with reference to FIG. 1. As shown in FIG. 2, the intensity modulation envelope of the illumination signal emitted from the signaling device 101 is shown as a sinusoidal wave shape, but is also typically modulated as a square wave. The signal is then reflected from various objects 21, 22, and 23 within the scene being imaged. As shown in FIG. 2, the light reflected back to the image sensor 102 will travel different distances based on the relative distances of the objects 21, 22, and 23 reflecting the light. This will result in the received light's modulation having different phases based on the distance traveled. Although not shown in FIG. 2, the amplitude of the waveform will also change based on the distance traveled and surface scattering properties of the objects.
FIG. 11 shows a schematic view of components employed to provide a known range sensing system. While this schematic diagram shows a shutter being used to modulate the image sensor, it is noted that various sensors may be used which do not require the use of a shutter for this purpose.
Methods for analyzing signal measurements are disclosed in, for example, U.S. Pat. No. 7,541,255 to Billmers et al., U.S. Pat. No. 7,391,505 to Dorrington, U.S. Pat. No. 5,835,199 to Phillips et al., and U.S. Pat. No. 5,534,993 to Ball et al., all of which are herein incorporated by reference in their entirety. Also, see International Publication No. 2004/090568 to Dorrington, International Publication No. 2006/031130 to Carnegie et al., and International Publication No. 2009/051499 to Payne et al., all of which are herein incorporated by reference in their entirety.
Conventional range imaging cameras, however, do not perform well in certain conditions. For example, when the air between a conventional range imaging camera and an object to be imaged is not clear, such as when fog or smoke is present, or where imaging is performed through glass or plastic windows, the conventional range imaging camera may not be able to accurately determine the distance to the object. In particular, particles or debris suspended in the air may result in the illumination signal from the conventional range imaging camera being partially reflected back, which may then result in an inaccurate distance determination.